Noun Phrase Recognition by System Combination

The performance of machine learning algorithms can be improved by combining the output of different systems. In this paper we apply this idea to the recognition of noun phrases. We generate different classifiers by using different representations of the data. By combining the results with voting techniques described in (Van Halteren et al., 1998) we manage to improve the best reported performances on standard data sets for base noun phrases and arbitrary noun phrases. 1 I n t r o d u c t i o n (Van Halteren et al., 1998) and (Brill and Wu, 1998) describe a series of successful experiments for improving the performance of part-of-speech taggers. Their results have been obtained by combining the output of different taggers with system combination techniques such as majority voting. This approach cancels errors that are made by the minority of the taggers. With the best voting technique, the combined results decrease the lowest error rate of the component taggers by as much as 19% (Van Halteren et al., 1998). The fact that combination of classifiers leads to improved performance has been reported in a large body of machine learning work. We would like to know what improvement combination techniques would cause in noun phrase recognition. For this purpose, we apply a single memorybased learning technique to data that has been represented in different ways. We compare various combination techniques on a part of the Penn Treebank and use the best method on standard data sets for base noun phrase recognition and arbitrary noun phrase recognition. 2 M e t h o d s and e x p e r i m e n t s In this section we start with a description of our task: recognizing noun phrases. After this we introduce the different data representations we use and our machine learning algorithms. We conclude with an outline of techniques for combining classifier results. 2.1 Task description Noun phrase recognition can be divided in two tasks: recognizing base noun phrases and recognizing arbit rary noun phrases. Base noun phrases (baseNPs) are noun phrases which do not contain another noun phrase. For example, the sentence In [ early trading ] in [ Hong Kong ] [ Monday ] , [ gold ] was quoted at [ $ 366.50 ] [ an ounce ] . contains six baseNPs (marked as phrases between square brackets). The phrase $ 366.50 an o u n c e is a noun phrase as well. However, it is not a baseNP since it contains two other noun phrases. Two baseNP data sets have been put forward by (Ramshaw and Marcus, 1995). The main data set consist of four sections (15-18) of the Wall Street Journal (WSJ) part of the Penn Treebank (Marcus et al., 1993) as training material and one section (20) as test material 1. The baseNPs in this data are slightly different from the ones that can be derived from the Treebank, most notably in the attachment of genitive markers. The recognition task involving arbitrary noun phrases at tempts to find both baseNPs and noun phrases that contain other noun phrases. A standard data set for this task was put forward at the CoNLL-99 workshop. It consist on the same parts of the Penn Treebank as the main baseNP data set: WSJ sections 15-18 as training data and section 20 as test data 2. The noun phrases in this data set are the same as in the Treebank and therefore the baseNPs in this data set are slightly different from the ones in the (Ramshaw and Marcus, 1995) data sets. In both tasks, performance is measured with three scores. First, with the percentage of detected noun phrases that are correct (precision). Second, with the percentage of noun phrases in the data that were found by the classifier (recall). And third, 1This (Ramshaw and Marcus, 1995) baseNP data set is available via ftp://ftp.cis.upenn.edu/pub/chunker/ 2Software for generating the data is available from http://lcg-www.uia.ac.be/conl199/npb/